Imaging apparatus

ABSTRACT

An image processing circuit includes a spectrum estimating portion for inputting image data, obtaining data required for spectrum estimation from an estimation data supplying portion and estimating spectrums of pixels, a scattering feature calculating portion for calculating several scattering features based on spectrums of pixels from the spectrum estimating portion and data required for feature calculation from the feature calculation data supplying portion, and a color image generating portion for performing a display color calculation based on a scattering feature image from the scattering feature calculating portion and for determining RGB values of respective pixels and outputting RGB images in order to display scattering features as a color image.

TECHNICAL FIELD

The present invention relates to an imaging apparatus for performingscattering imaging processing of a living body tissue.

BACKGROUND ART

It is known that many digestive tract tumor diseases such as anesophagus cancer are formed from a basal layer within epitheliums, whichis the most outer layer of a mucous membrane of a digestive tract. Asthe malignant degree increases, abnormal cells formed from the basallayer increase and, at last, the entire of epitheliums is replaced. Thetumor change in the epitheliums involves a cell simple variant andpathological changes in structure, that is so-called a structuralvariant. As a result, an irregular tissue arrangement is exhibited whichis different from a normal pathological image.

An object of an endoscopic diagnosis is to find this kind of tumor asearly as possible. Finding this kind of tumor at an earlier stage mayincrease the possibility for curing the tumor completely increases byperforming a less invasive operation such as an endoscopic treatment.

However, some kinds of tumor such as an esophagus cancer do not haveclear form (that is, a polyp or subsidence form) very much at an earlierstage and cannot be always found easily.

Many propositions have been made so far for finding and discriminatingtumors having poor changes in form at an earlier stage.

Scattering spectroscopy and scattering imaging (as disclosed in JapaneseUnexamined Patent Application Publication No. 2002-95635) is regarded asa leading technology among them. The scattering spectroscopy andscattering imaging are a technology for finding an early change which isdifficult to find on a general observation image by optically capturingthe scattering change based on a fact that nucleus and structuralvariants may cause an optical scattering change.

Conventionally, many propositions each using a polarizing optical systemhas been made in order to measure and/or image a scatteringcharacteristic of the inner part of epitheliums. While a rear singlelight scattered from the surface of the epitheliums holds a polarizedcomponent, multiple light scattered from the inner layer of theepitheliums (such as a mucous membrane layer or a mucous membrane innerlayer) are not polarized. Based on the knowledge and based ondifferential observation values of the horizontal and vertical polarizedcomponents, the scattering characteristic is imaged in the proposedtechnologies.

By illuminating a living body tissue with observation light polarized ina certain direction (such as the horizontal direction), the rearscattering light from a cell arrangement of the surface of theepitheliums can be observed as a polarized component in the samedirection (such as the horizontal direction). On the other hand, thelight propagated to the inner part of the epitheliums is not polarizeddue to the multiple scattering effect because of a structure on a celland/or various tissues can be observed as scattering light reflected bythe surface of the tissue.

By observing the light by using a polarizer in a different direction(such as the vertical direction) from that of the observed light, themagnitude of multiple scattered light can be estimated. The value isused to correct the influence of the multiple scattering included in theobserved light (horizontally polarized light) maintaining most polarizedlight by performing a differential operation and to extract singlescattered light from cells of the surface layer of the epitheliums.

The single scattering phenomenon from cells can be modeled as Miescattering from various spherical particles floating in protoplasm. Acharacteristic of Mie scattered light is that the scattering spectrumform depends on a size of a scattering particle, a refractive indexratio with respect to a peripheral medium (mainly protoplasm in thiscase) and a observation wavelength. Especially, the relationship betweenthe particle size and the spectrum form is important.

The particle size of the epithelium of the mucous membrane can beestimated by fitting the spectrum form of the single scattered lightextracted by the measurement of the polarized light by using a Miescattering model and by using different particle sizes and thenon-linear least square technique, for example.

A cell nucleus is considered as one of main elements contributing to thescattering in the epitheliums. Therefore, it is considered that theparticle size estimated by the technique has a high correlation with thesize of the cell nucleus.

Since the above-described nucleus variant involves a nucleus swelling(which means that the size of the nucleus increases from the normal sizewith the tumor changes), the estimation of the size of the nucleus byusing the technique allows the estimation of a state of the tumor changein the epitheliums.

Therefore, spectroscopy using polarized light and imaging have apossibility to image a nucleus swelling.

As described above, the spectroscopy and imaging by using polarizedlight may image a nucleus swelling quantitatively. However, theapplication to an endoscope may cause problems below:

A special scope self-containing a polarizing optical system is required;

A highly sensitive image pickup element (an optical element forgenerating and receiving polarized light) is required because polarizedlight is used (the light energy extremely decreases when a polarizer isused); and

A device is required for obtaining angles of illumination light andobservation light (rear scattering angles) precisely in order to modelbased on Mie scattering (in Mie scattering model, the angle of observedrear scattered light largely depends on the spectrum form).

By overcoming these problems, high-performance scattering imagingapparatus may be achieved. However, the problems still remain from theviewpoint of the cost of the apparatus.

The present invention was made in view of the problems. It is an objectof the invention to provide an imaging apparatus which can perform easyscattering imaging only by improving a light source device and theinternal part of a processor when an existing endoscopic optical systemis used as it is.

DISCLOSURE OF INVENTION

There is provided an imaging apparatus, including a light source device,an image pickup device for converting a living body observed image tovideo signals by using light irradiated from the light source device forobservation, and a processor for generating a living body image from thevideo signals, wherein the processor has means for generating a livingbody image having at least a scattering feature of a living body tissueas image information.

The other features and advantages of the present invention will besufficiently apparent from following descriptions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 to 10 relate to a first embodiment of the present invention.FIG. 1 is a configuration diagram illustrating a configuration of ascattering imaging apparatus; FIG. 2 is a diagram showing a structure ofa rotating filter in FIG. 1; FIG. 3 is a diagram showing a spectralcharacteristic of a rotating filter in FIG. 2; FIG. 4 is a block diagramillustrating a configuration of an image processing circuit in FIG. 1;FIG. 5 is a diagram for describing a mucous membrane tissue of adigestive tube on which the scattering imaging apparatus in FIG. 1performs scattering imaging; FIG. 6 is a diagram for describing a filterhaving a spectral characteristic causing a desired scatteringcharacteristic in the mucus membrane tissue of the digestive tube inFIG. 5; FIG. 7 is a diagram for describing an operation of a spectrumestimating portion in FIG. 4; FIG. 8 is a flowchart showing a flow ofprocessing by the spectrum estimating portion in FIG. 4; FIG. 9 is adiagram for describing an operation of a scattering feature calculatingportion in FIG. 4; and FIG. 10 is a diagram showing a structure of avariation example of the rotating filter in FIG. 1.

FIG. 11 is a block diagram showing a configuration of an imageprocessing circuit according to a second embodiment of the presentinvention.

FIGS. 12 and 13 relate to a third embodiment of the present invention.FIG. 12 is a configuration diagram showing a configuration of ascattering imaging apparatus; and FIG. 13 is a diagram showing astructure of a rotating filter in FIG. 12.

FIG. 14 is a diagram showing a body surface image pickup deviceaccording to a fourth embodiment of the present invention.

FIGS. 15 to 18 relate to a fifth embodiment of the present invention.FIG. 15 is a configuration diagram showing a configuration of an imageprocessing circuit; FIG. 16 is a first diagram for describing anoperation of the image processing circuit in FIG. 15; FIG. 17 is asecond diagram for describing an operation of the image processingcircuit in FIG. 15; and FIG. 18 is a flowchart for describing anoperation of the image processing circuit in FIG. 15.

FIGS. 19 to 23 relate to a sixth embodiment of the present invention.FIG. 19 is a configuration diagram showing a configuration of an imageprocessing circuit; FIG. 20 is a first diagram for describing anoperation of the image processing circuit in FIG. 19; FIG. 21 is asecond diagram for describing an operation of the image processingcircuit in FIG. 19; FIG. 22 is a third diagram for describing anoperation of the image processing circuit in FIG. 19; and FIG. 23 is aflowchart for describing an operation of the image processing circuit inFIG. 19.

BEST MODE FOR CARRYING OUT THE INVENTION

The present invention will be described in detail with reference toattached drawings.

First Embodiment

As shown in FIG. 1, an endoscopic device 1 included in a scatteringimaging apparatus according to this embodiment includes an electronicendoscope 3 having a CCD 2 as image pickup means to be inserted into abody cavity for picking up an image of a tissue in the body cavity, alight source device 4 for supplying illumination light to the electronicendoscope 3, and a video processor 7 for signal-processing image pickupsignals from the CCD 2 of the electronic endoscope 3 and displaying anendoscopic image on an observation monitor 5.

The light source device 4 includes a xenon lamp 11 for emittingillumination light, a heat ray cut filter 12 for shielding heat rays ofwhite light, an aperture device 13 for controlling a light amount ofwhite light through the heat-ray cut filter 12, a rotating filter 14 forchanging illumination light to field sequential light, a condensing lens16 for gathering field sequential light through the rotating filter 14onto an incident plane of a light guide 15 disposed in the electronicendoscope 3, and a control circuit 17 for controlling the rotation ofthe rotating filter 14.

As shown in FIG. 2, the rotating filter 14 has a disk shape and rotatesabout the center. The rotating filter 14 has a filter set of a C1 filter14 c 1, C2 filter 14 c 1 and C3 filter 14 c 1 for outputting fieldsequential light having spectral characteristic as shown in FIG. 3. Asshown in FIG. 1, a rotating filter motor 18 is drive-controlled by thecontrol circuit 17 so that the rotating filter 14 can rotate.

Power is supplied from a power source portion 10 to the xenon lamp 11,the aperture device 13 and the rotating filter motor 18.

Referring back to FIG. 1, the video processor 7 includes a CCD drivingcircuit 20, an amplifier 22, a process circuit 23, an A/D converter 24,a white-balance circuit 25, a selector 26, synchronized memories 27, 28and 29, an image processing circuit 30, D/A circuits 31, 32 and 33, atiming generator 35, and a dimmer circuit 43. The CCD driving circuit 20drives the CCD 2. The amplifier 22 amplifies image pickup signals whichpick up the image of a tissue in a body cavity by using the CCD 2through an objective optical system 21. The process circuit 23 performscorrelation double sampling and noise removal on image pickup signalshaving passed through the amplifier 22. The A/D converter 24 convertsthe image pickup signals having passed through the process circuit 23.The white-balance circuit 25 performs white-balance processing on imagedata from the A/D converter 24. The selector 26 and the synchronizedmemories 27, 28 and 29 are used for synchronize field sequential lightby the rotating filter 14. The image processing circuit 30 performsreading gamma correction processing, contour emphasis processing, colorprocessing and so on on image data of field sequential light stored inthe synchronized memories 27, 28 and 29. The D/A circuits 31, 32 and 33convert image data from the image processing circuit 30 to analogsignals. The timing generator 35 inputs synchronization signals insynchronization with a rotation of the rotating filter 14 from thecontrol circuit 17 of the light source device 4 and outputs differentkinds of timing signals to the respective circuits. The dimmer circuit43 inputs image pickup signals having passed through the process circuit23, controls the aperture device 13 of the light source device 4 andperforms proper brightness control.

As shown in FIG. 4, the image processing circuit 30 includes a spectrumestimating portion 52, a scattering feature calculating portion 54, anda color image creating portion 55. The spectrum estimating portion 52estimates a spectrum of respective pixels by inputting image data fromthe synchronized memories 27, 28 and 29 and obtaining data required forthe spectrum estimation from an estimation data supplying portion 51.The scattering feature calculating portion 54 calculates severalscattering features based on the spectrum of respective pixels from thespectrum estimating portion 52 and data required for feature calculationfrom the feature calculation data supplying portion 53. The color imagecreating portion 55 calculates a display color based on a scatteringfeature image from the scattering feature calculating portion 54 anddetermines an RGB value of respective pixels so as to display ascattering feature as a color image. Thus, the color image creatingportion 55 outputs the RGB values as an RGB image to the D/A circuits31, 32 and 33.

The estimation data supplying portion 51 and the feature calculationdata supplying portion 53 are provided in the video processor 7 or in anexternal block.

A mucous membrane tissue of a digestive tube such as the esophagusalmost has a structure as shown in FIG. 5. A tumor such as an esophagealcancer emerges from a basal layer separating the epitheliums and themucous membrane layer. The tumor emerging from the basal layer has anucleus variant and a structure variant and replaces the entire ofepitheliums by variant cells. Then, the tumor advances to a cancerthrough a so-called variant forming state.

The epitheliums include a flat epithelium and exhibit a strongscattering characteristic due to the hard-grained cell structure. Thescattering characteristic has a wavelength dependence and may have acharacteristic that the scattering characteristic decreases from theshort wavelength to the long wavelength (therefore, it is consideredthat most of light with a short wavelength is scattered and reflected inthe epitheliums and the deeper transmission to the mucous membrane layerand lower layers may be less.)

Accordingly, in order to capture a change in scattering characteristicwithin epitheliums, short-wavelength light is more suitable thanlong-wavelength light. This is true for imaging, and narrow bandmultiband illumination will be described, for example.

As shown in FIG. 6, in order to naturally reproduce colors in a rotatingfilter for generating field sequential light, combinations of wide bandfilters such as C-B, C-G and C-R are generally used. The rotating filterrotates fast so that three band lights can illuminate an objectchronologically and sequentially images the object by using a monochromeCCD. The images are synthesized by a video processor, and the bandimages corresponding to the respective illumination lights are assignedto Blue, Green and Red channels of an observation monitor. Thus, onecolor image is displayed.

If a filter having a desired characteristic can be produced, thecontrast of an image of the blood vessel can be improved by changing thehalf-value breadth to a narrow band without a large change in centerwavelength of the C-B, C-G and C-R such as C2, C4 and C6.

However, the rotary filter 14 selects bands from short-wavelength lighthere as described above for scattering imaging within the epitheliums.C1, C2 and C3 in FIG. 6 correspond thereto. The multiple band imageswith shorter wavelengths may show a structure within the epitheliumsbetter than those of bands (C4, C5, C6 and C7) with longer wavelengths.

Here, the band images corresponding to C1, C2 C3 illumination lights maybe assigned to the Blue, Green and Red channels so that one color imagecan be reproduced therefrom on an observation monitor. However, whatkind of scattering characteristic the color information corresponds toand what kind of pathological change (such as a degree of a structuralvariant and a degree of a nucleus swelling) the color change correspondsto are not clear, and understanding the image is not easy. Furthermore,even when a doctor using an endoscope observes such an image during anexamination, it is difficult for the observation to contribute to theearly discovery of a tumor.

According to this embodiment, a spectral reflectance is estimated fromthese multiple band images, and a change in feature amount in an imageis displayed as color information by converting the estimated spectralreflectance to a feature having higher pathological correlation based onan optical model of a living body tissue.

It is an object of this embodiment to estimate spectral reflectances ofpixels from a narrow band multiband image, estimate a feature amounthaving higher pathological correlation based on an optical model andgenerate color information based on a change in the estimated featureamount in an image.

[Operation]

Narrow-band band images (for example, three band corresponding to theshort wavelength bands, C1, C2 and C3, as shown in FIG. 6) output fromthe synchronized memories 27, 28 and 29 are input to the spectrumestimating portion 52 provided in the image processing circuit 30. Thespectrum estimating portion 52 obtains data required for the spectrumestimation from the estimation data supplying portion 51 within theimage processing portion or in an external block. Thus, the spectrumestimating portion 52 estimates the spectrum of each pixel.

The estimated spectrums, that is, the spectrum images are values to beinput to the scattering-feature calculating portion 54. Thescattering-feature calculating portion 54 obtains data required for thefeature calculation from the feature-calculation data supplying portion53 provided in the image processing portion or in the external block.Thus, the scattering feature calculating portion 54 calculates severalscattering features. At this time, several scattering features areassigned to the respective pixels.

The scattering-feature calculating portion 54 outputs the scatteringfeature images to the color image generating portion 55. The color imagegenerating portion 55 performs a display color calculation based on thescattering feature images. Then, in order to display scattering featuresas a color image, RGB values of the respective pixels are determined andare output to the D/A circuits 31, 32 and 33 as the RGB images.

Next, an operation in each of the blocks (including the spectrumestimating portion 52, the scattering-feature calculating portion 54 andthe color image creating portion 55) will be described. The detail isdisclosed in a document, “V. Backman, R. Gurjar, K. Badizadegan, I.Itzkan, R. R. Dasari, L. T. Perelman, and M. S. Feld, ‘Polarized LightScattering Spectroscopy for Quantitative Measurement of EpithelialCellular Structures In Situ,’ IEEE J. Sel. Top. Quantum Electron, 5,1019-1026 (1999)”. In equations below, the symbol, “ˆ” indicates avector (with a lowercase letter) and matrix (with an uppercase letter)having several elements.

[Spectrum Estimating Portion 52]

A relationship between object spectral reflectances and pixel values ofmultiband images is represented by EQ1, which is an imaging equation:$\begin{matrix}{{g^{\hat{}} = {{H^{\hat{}}f^{\hat{}}} + n^{\hat{}}}}{{g^{\hat{}} = \begin{bmatrix}g_{c\quad 1} \\g_{c\quad 2} \\g_{c\quad 3}\end{bmatrix}},{H^{\hat{}} = \begin{bmatrix}h_{11} & h_{12} & \cdots & h_{1L} \\h_{21} & h_{22} & \cdots & h_{2L} \\h_{31} & h_{32} & \cdots & h_{3L}\end{bmatrix}},{f^{\hat{}} = \begin{bmatrix}f_{1} \\f_{2} \\\vdots \\f_{L}\end{bmatrix}},{n^{\hat{}} = \begin{bmatrix}n_{c\quad 1} \\n_{c\quad 1} \\n_{c\quad 2}\end{bmatrix}}}} & (1)\end{matrix}$

Here, gˆ is a pixel value column vector having dimensions (N: three inthis embodiment) the number of which is equal to the number of bands. fˆis an object spectral reflectance column vector, and the value isdiscrete at L in the wavelength direction. nˆ is a noise column vector.Hˆ is a system matrix of L×N having N row vectors, which is a spectralsensitivity characteristic of each band.

The problem is that Hˆ is known and the spectral reflectance of anobject is estimated from the observation value gˆ. Hˆ is known asspectral characteristics of an imaging system, such as an observationlight spectrum, a spectral transmittance characteristic of a narrow bandfilter and a spectral sensitivity characteristic of an image pickupelement.

In general, since “number of bands N<number of samplings L”, theestimation problem is ill-condition. That is, an infinite number of fˆsatisfying EQ1 exist for gˆ. (In other words, since the number ofequations is lower than the number of unknown values, various solutionsare possible. Therefore, one solution cannot be determined as far assome condition is given).

By preparing foresight information and limiting a solution space(L-dimensional spectrum space in this case), (this would be a conditionto determine one exclusive solution), an appropriate estimated solutionmust be found. That is, it is concluded in a problem for obtaining anoptimum solution within a partial space, of the L-dimensional spectrumspace by using foresight information, in which spectrums distributecandidates of the solution.

Wiener estimation is generally used as a technique using foresightinformation. EQ2 represents an estimation matrix A in Wiener estimation.By multiplying the estimation matrix A from the right of the observationvector g, the spectrum is estimated. Therefore, the spectrum estimatingmeans operates as a matrix calculator using the predefined estimationmatrix A.i Aˆ=R_(f) ˆH ^(r)ˆ(HˆR _(f) ˆH ^(r) e,cir +R_(n)ˆ)⁻¹   (2)

Here, R_(f)ˆ is an auto-correlation matrix (L×L) in the wavelengthdirection of the object spectrum to be estimated, and R_(n)ˆ is anauto-correlation matrix of additive noise appearing as nˆ in EQ1. R_(n)ˆcan be estimated from the pre-measured noise characteristic of animaging system (which is a total system having a combination of a lightsource and a scope) and is known. The foresight information here isR_(f)ˆ and R_(n)ˆ. Especially, R_(f)ˆ is the most important parameterinfluencing on the validity of the spectral reflectance to be estimated.

Conventionally, the opposite of a differential operator matrix (that is,low-frequency enhancing filter in a space frequency area) is often usedas the auto-correlation matrix R_(f)ˆ given that the spectrum to beestimated is smooth in the wavelength direction (that is, the spectrumhas a smoother characteristic in the wavelength direction without rapidchanges in wavelength unlike an emission spectrum). Alternatively, aMarcov transition matrix is often used as the auto-correlation matrixR_(f)ˆ since the spectral reflectance can be represented by a statisticsmodel such as Marcov model. According to this embodiment, anauto-correlation matrix is used which is obtained from a spectrumsestimated from a discrete particle structure model (which is calledoptical model hereinafter and will be described as the auto-correlationmatrix R_(f)ˆ later) of a living body tissue.

Next, the optical model will be described. A living body tissue includesvarious elements such as a fiber tissue, cells, lymphocytes, capillarytubes, nucleuses and small organs within each cell.

Since a scattering occurs at an area having a large change in refractiveindex, a main scattering body (scattering entity) in the living bodytissue is considered to be a small organ within a cell such as a nucleusand mitochondria. The phase function and scattering coefficient of aparticle having a wavelength equivalent to or a little smaller than theobservation wavelength can be estimated by using the Mie scatteringmodel. The phase function represents a probability of the scattering oflight incident on the scattering entity from a direction s to adirection s′. The scattering coefficient is a parameter indicating thenumber of times that a photon is subject to scattering for everydistance unit.

The Mie model has 2πma/λ as a parameter of the model (where λ is awavelength, m is a refractive index ratio and a is a diameter of thescattering entity. Since the refractive index ratio between the nucleusand the protoplasm may not have a large change, Mie scattering can be amodel for estimating a scattering spectrum by mainly using thescattering entity as a parameter.

On the other hand, from the information (particle size distributionfunction) of the size and density of particles (such as nucleuses andsmall organs in cells) in the living body tissue, the phase function andscattering coefficient can be estimated by using the Mie scatteringmodel. FIG. 7 shows a conceptual diagram of a particle sizedistribution. An actual diameter of a particle is considered to be inthe range from about 0.4 μm for a small organ within a cell to about 4μm for a nucleus. As the structure variant advances from the normaltissue, the particle size distribution is considered to change (f1(d) tof2(d) where d is a particle diameter) as indicated by an arrow in FIG.7. The phase function and the scattering coefficient are calculated byusing the Mie scattering model based on the particle size distributionfunction and the refractive index rate between a particle and aperipheral medium (about 1.03 where the peripheral medium should beprotoplasm). The particle size distribution function can be applied to anormal distribution and/or a logarithm normal distribution. An opticalcoefficient is calculated from the Mie scattering model for a change inparticle size distribution parameter (such as a mean and a standarddeviation), which is conceivable for a target. A spectrum is calculatedby simulating a multiple scattering process by using a light propagationmodel based on the calculated optical coefficient.

The light propagation model may be a method using a scattering equationwhich is advantageous in calculation time as an analytical method buthas a large limitation in degree of freedom in model shape.Alternatively, the light propagation model may be Montecarlo modelrequiring a time for the calculation but having a higher degree offreedom in model form. In this way, different kinds of methods can beused in accordance with the condition.

Summing up the model calculation up to this point, as shown in FIG. 8,particle size distribution parameters (including a mean and a standarddeviation) are obtained at a step S1. The particle size distributionparameters are input to Mie scattering model at the step S2. At a stepS3, a scattering coefficient and a phase function are output from theMie scattering model. In reality, the scattering coefficient and thephase function are calculated by applying the Mie scattering model forevery size of each particle. Then, a Mie scattering calculation inconsideration of the particle size distribution by using the particlesize as a weighted sum mean, which is a weight function.

Through the model calculation, a spectrum is calculated by using anucleus variant and structure variant with the tumor change withinepitheliums as changes in particle size distribution parameters (meanand standard deviation). Then, a solution space (spectrum space) islimited by using the spectrum as preliminary information. In otherwords, the auto-correlation matrix R_(f)ˆ in Wiener estimation iscalculated in advance from the spectrum distribution estimated from themodel calculation.

More specifically, at a step S4, a spectrum change in accordance withchanges in particle size distribution parameters (changes in mean andstandard deviation) obtained from the pathological information iscalculated by using Mie scattering model and a light propagation model.As a result of the calculation, a spectrum distribution in accordancewith the changes in particle size distribution parameters is formed in aspectrum space at a step S5. By using the spectrum distribution as apopulation, the auto-correlation matrix in the wavelength direction ofthe spectrum is estimated.

As described above, the scattering spectrum within the epitheliums isestimated by using the auto-correlation matrix estimated in advance fromthe optical model (particle size distribution model+Mie scatteringmodel+light propagation model). Therefore, the matrix A to be calculatedby EQ2 based on Hˆ, R_(f)ˆ and R_(n)ˆ is stored in the estimation datasupplying portion.

[Scattering Feature Calculating Portion 54]

Various features can be calculated from the spectrum estimated by thespectrum estimating portion 52. This embodiment focuses on the particlesize distribution parameters and proposes a method for estimating afeature amount correlated to the parameters from the spectrum. FIG. 9shows a conceptual diagram thereof.

When the auto-correlation matrix is estimated, a spectrum change rangefor the changes in particle size distribution parameters (including meanand standard deviation) are known.

Therefore, feature axes corresponding to the mean and standarddeviation, which is considered to be scattering features, are known asF1 and F2. In other words, spectrums are distributed in a partial spacewhich locates between F1 and F2. Thus, from the calculated spectrum,scattering features are obtained including projection values (f1 and f2)to F1 and F2 and the brightness (such as an area of the spectrum) as athird value. Therefore, spectrums of the feature axes are stored in thefeature calculation data supplying portion. The calculation in thecalculating portion is an inner product calculation of a feature axisspectrum and a scattering spectrum.

[Color Image Generating Portion 55]

A color image is generated by assigning the scattering features and thespectrum brightness output from the scattering feature calculatingportion 54 to the Blue, Green and Red channels. Here, the pixel valuesmust be properly quantized to 8 bits, for example, in accordance withthe D/A performance of a downstream unit. In order to discover a changeat an early stage on a screen, the relative scattering changes, that is,the degrees of the nucleus variant and structure variant may be onlyrequired. Therefore, in a frame, the scattering features are quantizedat a predetermined level such as 8-bits by calculating the dynamic rangeand are output as RGB signals.

Summing up the characteristics of this embodiment, a scattering spectrumis estimated by using a multiband image in a short wavelength range,which is conceivable as having a lower invasive degree and stronglyreflecting features of the inside of the epitheliums and anauto-correlation matrix estimated from a model-based spectralreflectance distribution calculated from a Mie scattering model and alight propagation model by modeling the epithelial tissue as a discreteparticle structure. The projection values to feature axes correspondingto pre-calculated particle size distribution parameters in a spectrumspace are used as scattering features. Then, the amounts of thesefeatures for each pixel are assigned to color channels, and thescattering imaging is achieved by using the color information.

[Advantages]

According to this embodiment, an imaging correlated to changes inscattering characteristic is allowed by performing calculations in anarrow-band filter and a processor without a special scope such as apolarizing optical system. Thus, this embodiment allows the visualrecognition of a feature such as a structure variant within epitheliums,which cannot be observed easily before.

In order to obtain a general observation image, the rotating filter 14,as shown in FIG. 10, having narrow-band filters 14C1 to C6 for themultiple narrow bands C1 to C6 (see FIG. 6) may be used. In this case,memories number of which is equal to the number of filters are providedfor respective images of the narrow-band filters. Furthermore, in thiscase, the image processing circuit 30 includes a general observationimage generating portion and the spectrum estimating portion 52+thescattering feature calculating portion 54 (not shown) according to thefirst embodiment. The image processing circuit 30 further includes acontrast enhancing coefficient calculating portion for calculating anenhancement coefficient based on an output of the scattering featurecalculating portion 54. A quantized value is calculated based on a valuecorrelated to a synthesized feature using a mean or a standard deviationand mean of a particle size distribution, for example, within a frame.Based on the value, an enhancement coefficient of a space frequency iscalculated for a luminance channel of an image generated by the generalobservation image generating portion. Thus, the contrast enhancement canbe performed on the general observation image based on the scatteringfeatures.

Second Embodiment

Since a second embodiment is substantially the same as the firstembodiment, only differences therebetween will be described. The samereference numerals are given to the same components as those of thefirst embodiment, and the description will be omitted here.

[Structure and Operation]

In order to estimate a scattering spectrum in epitheliums, multiplenarrow-band filter is used in a short-wavelength area. The band has astrong scattering characteristic while the band has a maximum absorption(415 nm) of hemoglobin. For example, an esophagus flat epithelium doesnot have capillary tubes in the normal mucous membrane. With theswelling of a nucleus, the blood vessel within a nipple may expandand/or a vascularity may occur within the epitheliums. This kind ofcapillary tube image has a unique spectrum due to hemoglobin, which maybecome an error factor in the scattering spectrum estimation. Therefore,the capillary tube image as an absorption image is separated before thescattering spectrum estimation.

For the separation, the facts are considered that these capillary tubeimages dynamically have high spatial frequency and that a scatteringimage itself forms a low frequency image due to multiple scatterings.More specifically, as shown in FIG. 11, the image processing circuit 30according to this embodiment includes filtering portions 61, 62 and 63corresponding to narrow-band band images by spatial frequency filterbefore the scattering spectrum estimating portion 52. The operations bythe filtering portions 61, 62 and 63 can be performed by a convolutioncomputer having an FIR filter. The convolution computer includes a highfrequency band pass filter for separating capillary tube images and alow pass filter for estimating scattering spectrums.

Outputs from the filtering portions 61, 62 and 63 corresponding to therespective narrow-band band images are separated into high frequencyimages C1H, C2H and C3H (where H is a subscript) and low frequencyimages C1L, C2L and C3L (where L is a subscript) corresponding to therespective band images. The low frequency images are output to thescattering spectrum estimating portion 52. The high frequency images areoutput to capillary tube image generating means 64.

As described in the first embodiment, the scattering spectrum estimatingportion 52 estimates a scattering spectrum by calculating anauto-correlation matrix in Wiener estimation from a rear scatteringspectrum distribution within the epitheliums, which is estimated from adiscrete particle structure model.

On the other hand, the capillary tube image generating portion 64generates a capillary tube image more clearly by properly removing noisefrom and, in some cases, using a matched filter modeling a blood vesselstructure for the high frequency images generated from the bands. Then,the capillary tube images are output to an image signal generatingportion 65 as luminance information.

The image signal generating portion 65 generates a scatteringcharacteristic by using a color map based on the output from thescattering spectrum estimating portion 52 and synthesizes the capillarytube images as luminance information. Thus, the image signal generatingportion 65 outputs the scattering+capillary tube absorption images to anobservation monitor 5.

[Advantages]

According to this embodiment, an imaging correlated to changes inscattering characteristic is allowed by performing calculations in anarrow-band filter and a processor without a special scope such as apolarizing optical system. Thus, this embodiment allows the visualrecognition of a feature such as a structure variant within epitheliums,which cannot be observed easily before. By separating capillary tubeimages, which are absorption images, in advance by using spatialfiltering means, the decrease in estimation precision of spectralreflectance can be prevented. At the same time, the capillary tubepattern and scattering images, which are important for discriminationdiagnosis, can be synthesized and displayed.

Third Embodiment

A third embodiment is substantially the same as the first embodiment.Therefore, only the differences therebetween will be described. The samereference numerals are given to the same components, and thedescriptions will be omitted.

[Structure and Operation]

As shown in FIG. 13, the rotating filter 14 according to this embodimenthas a disk shape and has a double structure rotatably about the center.A first filter set including a C1 filter 14C1, a C2 filter 14C2 and a C3filter 14C3 for outputting narrow band field sequential light havingspectral characteristics indicated by C1 to C3 shown in FIG. 6 aredisposed on the outer diameter part. A second filter set including a C4filter 14C4, a C5 filter 14C5 and a C6 filter 14C6 for outputting fieldsequential light having spectral characteristics indicated by C4 to C6shown in FIG. 6 are disposed on the inner diameter part.

As shown in FIG. 12, the rotating filter 14 is rotated under the drivingcontrol of the rotating filter motor 18 by the control circuit 17. Themovement in the diameter direction (which is movement vertical to anoptical path of the rotating filter 14 and selectively moves the firstfilter set or second filter set of the rotating filter 14 on the opticalpath) is performed by a mode switching motor 19 in accordance with acontrol signal from a mode switching circuit 42 in the video processor7.

Power is supplied from the power source portion 10 to the xenon lamp 11,the aperture device 13, the rotating filter motor 18 and the modeswitching motor 19.

The electronic endoscope 2 includes a mode select switch 41. The outputof the mode select switch 41 is output to the mode switching circuit 42in the video processor 7. The mode switching circuit 42 in the videoprocessor 7 outputs control signals to the dimmer circuit 43, a dimmingcontrol parameter switching circuit 44 and the mode switching motor 19of the light source device 4.

The dimming control parameter switching circuit 44 outputs a dimmingcontrol parameter in accordance with the first filter set or secondfilter set of the rotating filter 14 to the dimmer circuit 43. Thedimmer circuit 43 controls the aperture device 13 of the light sourcedevice 4 based on the control signal from the mode switching circuit 42and the dimming control parameter from the dimming control parameterswitching circuit 44. Thus, the brightness is controlled properly.

[Advantages]

According to this embodiment, in addition to the advantages of the firstembodiment, the observation in a body cavity with general observationlight can be performed by using the C4 filter 14C4, the C5 filter 14C5and the C6 filter 14C6.

Fourth Embodiment

Since a fourth embodiment is substantially the same as the firstembodiment, only differences therebetween will be described. The samereference numerals are given to the same components, and the descriptionwill be omitted here.

[Structure and Operation]

According to the above-described embodiments, an image pickup device isprovided within an endoscope, and scattering imaging of a tissue withina body cavity is performed. According to this embodiment, a scatteringimaging apparatus will be described which can irradiate narrow bandlight on a body surface and can detect a skin cancer.

As shown in FIG. 14, according to this embodiment, instead of theelectronic endoscope 2, a body surface imaging apparatus 84 is provided.The body surface imaging apparatus 84 has at its distal end portion ahood 81 to be touched to the skin, and at a distal end surface of thedistal portion a plane of light injection on a light guide 82 disposedin a ring shape and a plane of incidence on an image pickup portion 83including an objective optical system and a CCD. The hood 81 is madecontact with the skin, and field sequential light beams in narrow bandhaving spectral characteristics C1 to C3 are irradiated from the lightsource device 4 through the light guide 82. The image pickup portion 83picks up an image, and the image pickup signals are transmitted to thevideo processor 7.

[Advantages]

According to this embodiment, the same operational advantages as that ofthe first embodiment can be obtained even on the body surface, and askin cancer or the like can be detected.

Fifth Embodiment

Since a fifth embodiment is substantially the same as the firstembodiment, only differences therebetween will be described. The samereference numerals are given to the same components, and the descriptionwill be omitted here.

A mucous membrane of the digestive tract such as the esophagus has alayer structure. An early cancer mainly emerges and expands within asurface layer. Therefore, in order to find a cancer at an earlier stage,a pathological change occurring within the surface layer of the mucousmembrane must be captured as a picture.

However, in general, light reflected from a living body reacts withchanges in layers (second and lower layers) under the surface layer(first layer) because the surface layer of the mucous membrane issignificantly thin. The changes are scatterings and/or absorptions. Morespecifically, the changes are a pathological structure and/or thedensity of the blood vessel.

Therefore, an algorithm is required which is free from influences on thechanges in optical characteristic of the second layer as few as possibleand is used for calculating an amount of the scattering feature foremphasizing an optical characteristic (change in scattering) of thefirst layer.

According to the fifth embodiment, a mapping to the amount of scatteringfeature is obtained from a spectral image value or an observation value,which is a multiband image value.

[Structure]

As shown in FIG. 15, in the image processing circuit 30, stomach mucousmembrane mapping data, esophagus mucous membrane mapping data and so onas digestive tract mucous membrane mapping data 100 for each organ,which will be described later, obtained by multiple discriminationanalysis are stored in the feature calculation data supplying portion53. The scattering feature calculation portion 54 reads thecorresponding digestive tract mucous membrane mapping data 100 based onan organ select signal from input means (not shown) from the featurecalculation data supplying portion 53. Then, a scattering feature iscalculated.

Next, the digestive tract mucous membrane mapping data 100 to be storedin the feature calculation data supplying portion 53 will be described.While the structure of the mucous membrane tissue of a digestive tractsuch as the esophagus is illustrated in FIG. 5, here, when theepitheliums are a first layer and the entire layers including the basallayer and lower layers are a second layer, the influence of the secondlayer is strongly reflected on the spectrum estimated in the spectrumestimating portion 52 of the image processing circuit 30. The change inspectrum due to the swelling of the nucleus of the first layer, forexample, is masked.

According to this embodiment, a mapping to a partial space having asmaller influence of the second layer and enhancing the scatteringfeature of the first layer in the observation spectrum space isobtained. The obtained mapping is stored in the feature calculation datasupplying portion 53 as the digestive tract mucous membrane mapping data100.

By using the publicly known multiple discrimination analysis, such amapping can be obtained as a linear mapping for maximizing a spectrumchange depending on the characteristic change, which is the scatteringfeature in this case, of the first layer under a condition forminimizing a variation depending on the characteristic change of thesecond layer.

For example, it is assumed that a spectrum on the living body tissue isdistributed in a spectrum space as shown in FIG. 16. Here, a classrefers to a set of data having a same scattering feature of a layer,which is a target, such as the epitheliums of an early esophagus cancer.In FIG. 16, two classes are shown and refer to data sets having twokinds of epitheliums having different scattering features.

Each class includes an extension of data in accordance with changes inscattering and absorption characteristics of the layers other than theepitheliums. A mapping to the multiple discrimination space means aconversion for maximizing a distance between the classes under acondition for minimizing the extension within the classes shown in FIG.16 (and is known as Fisher linear identification when the mapping withthe two classes is linear).

By using this mapping, the spectrum of the living body tissue in FIG. 16is mapped to a space where a distance between the classes (inter-classdistribution) is maximized as shown in FIG. 17 and a variation withineach of the classes is minimized (intra-class distribution). Theintra-class distribution and the inter-class distribution can becalculated by a light scattering simulation, for example.

In other words, a scattering characteristic of a living body tissue of alayer, which is a target, such as the epithelium is enhanced when theinfluence of the absorption/scattering characteristic of the otherlayers is minimized.

[Operation]

According to this embodiment, as shown in FIG. 18, when respective imagedata are input from the synchronized memories 27, 28 and 29 in the imageprocessing circuit 30 at a step S51, the spectrum estimating portion 52obtains living body spectrum auto-correlation data from the estimationdata supplying portion 51 and estimates spectrums of respective pixelsat a step S52. At a step S53, the scattering feature calculating portion54 reads from the feature calculation data supplying portion 53 thedigestive tract mucous membrane mapping data 100 for mapping theintra-class distribution and the inter-class distribution to anoptimized space in accordance with an organ select signal, and ascattering feature is calculated. At a step S54, the color imagegenerating portion 55 performs a display color calculation based on thescattering feature image from the scattering feature calculating portion54. Then, in order to display the scattering feature as a color image,RGB values of the respective pixels are determined and are output to theD/A circuits 31, 32 and 33 as RGB images.

When the scattering feature space obtained by the multiplediscrimination analysis is three-dimensional, the color image generatingportion 55 assigns the respective axes included in the scatteringfeature space to RGB color channels. The maximum and minimum arepreviously determined for each of the axes, and the ranges of the colorchannels are assigned within the range.

In another color assigning method, information is assigned such that thecontrast in an image can be maximum. An image is input to a computer inframe (or in field), and data is mapped to the scattering feature spacewithin a screen. In accordance with the number of pixel values withinthe screen, data are distributed within the scattering feature space.The maximum distribution direction should be a direction having thedirection to which the changes in scattering feature are reflected most.Therefore, by using a general method such as KL expansion, a mappingvalue to the maximum distribution axis is obtained. One point on themaximum distribution axis is determined as a reference point, and acolor is assigned to the image in accordance with the distancetherefrom. An expression having the best visual recognition is adoptedlike assigning a color in a hue direction.

Instead of the estimation of an object spectrum from a multiband image,a value resulting from the correction of a gain valance with respect toa multiband image value may be used. In this case, the spectrumestimating portion 52 is not required. In the gain balance correctingmethod, an object having a known spectral reflectance, such as a whiteplate, is imaged, and a gain correction is performed such that thestrength ratio between the observed multiband image values can be aratio calculated from the spectral product of the band characteristicsand the known object spectral reflectance.

[Advantages]

According to this embodiment, in addition to the advantages of the firstembodiment, the scattering characteristics of a living body tissue of alayer, which is a target, such as the epithelium is enhanced byminimizing the influences of the absorption/scattering characteristicsof the other layers. Thus, the visual recognition characteristic isimproved.

Sixth Embodiment

Since a sixth embodiment is substantially the same as the fifthembodiment, only differences therebetween will be described. The samereference numerals are given to the same components, and the descriptionwill be omitted here.

[Structure]

As shown in FIG. 19, the image processing circuit 30 includes a bloodvessel structure extracting portion 111 and a mapping updating portion112. The blood vessel structure extracting portion 111 extracts bloodvessel structure information of a second layer (that is the entire lowerlayers including the basal and lower layers in FIG. 5) from image datafrom the synchronized memories 27, 28 and 29. The mapping updatingportion 112 calculates a variation within a class (intra-classdistribution) based on the blood vessel structure information extractedby the blood vessel structure extracting portion 111 and updatesdigestive tract mucous membrane mapping data 100 stored in the featurecalculation data supplying portion 53 based on the intra-classdistribution.

[Operation]

A living body tissue having a layer structure, such as an esophagusmucous membrane may have a unique blood vessel structure in which thesurface layer is a capillary tube and the medium deep layer is arelatively thicker blood vessel as shown in FIG. 20. When a living bodytissue having this kind of structure is observed by using bands havingdifferent center wavelengths, the blood vessel of the surface layer andthe blood vessel of the medium deep layer are reproduced on the shortwavelength side and on the long wavelength side, respectively, as shownin FIGS. 21 and 22 (see Japanese Unexamined Patent ApplicationPublications No. 2002-95635, 2002-34893 and 2002-34908).

Therefore, in the data set, the characteristics of the epitheliums arethe same between the position with the blood vessel and the positionwithout the blood vessel while the other layers have differentcharacteristics. The intra-class distribution can be estimated from thedata set.

As shown in FIG. 23, at a step S71, the blood vessel structureextracting portion 111 extracts a position of the blood vessel from theRGB images by using an image (such as the B image) on the longwavelength side. A general method such as threshold processing andspatial frequency filtering processing may be applied to the extractionof the position of the blood vessel. At a step S72, the mapping updatingportion 112 gathers many pixels at positions of the blood vessel andpixels excluding the blood vessel. At a step S73, the intra-classdistribution is calculated, and the digestive tract mucous membranemapping data 100 stored in the feature calculation data supplyingportion 53 is updated based on the intra-class distribution.

In this case, the blood vessel structure extracting portion 111 extractsthe position of the blood vessel from the RGB images by using the image(such as the B-image) on the long wavelength side from the RGB images.However, in order to obtain a band image for extracting the position ofthe blood vessel, an observation area may be illuminated by using aspecial band filter.

[Advantages]

Also according to this embodiment, the same advantages can be obtainedas those of the fifth embodiment.

According to this embodiment, illumination light is separated in band onthe light source side and is irradiated in order to obtain multibandimages. However, the structure is not limited thereto. Multiband imagesmay be obtained by using a band separating filter on the imagepicking-up side.

It is apparent that a wide variety of embodiments according to thepresent invention can be constructed widely based on the presentinvention without departing from the spirit and scope of the invention.The present invention is only limited by appended claims and is notlimited by specific embodiments thereof.

INDUSTRIAL APPLICABILITY

As described above, an imaging apparatus according to the presentinvention is useful as an apparatus for imaging a scattering feature inan internal tissue as image information.

1. An imaging apparatus, comprising: a light source device; an imagepickup device for converting a living body observed image to videosignals by using light irradiated from the light source device forobservation; and a processor for generating a living body image from thevideo signals, wherein the processor has means for generating a livingbody image having at least a scattering feature of a living body tissueas image information.
 2. The imaging apparatus according to claim 1,wherein the image pickup device is an endoscope.
 3. The imagingapparatus according to claim 1, wherein the light source deviceirradiate at least one band light beam.
 4. The imaging apparatusaccording to claim 1, wherein at least one of the band light beamsexists in a band positioned as blue light in a visual light wavelengthrange.
 5. The imaging apparatus according to claim 1, wherein theprocessor has means for estimating, from at least one living body image,spectrums corresponding to positions and/or an area in the image.
 6. Theimaging apparatus according to claim 5, wherein the means for estimatingthe spectrums have at least one or more matrix computers.
 7. The imagingapparatus according to claim 6, wherein a coefficient of the at leastone matrix computer determines a range having spectrums estimated in aspectrum space with at least one discrete wavelength.
 8. The imagingapparatus according to claim 7, wherein a coefficient of the at leastone matrix computer is designed by using a light propagation modelexpressing propagation of light in a scattering medium.
 9. The imagingapparatus according to claim 1, wherein the processor has means forestimating a scattering feature by a living body tissue from spectrumscorresponding to positions and/or areas in an image.
 10. The imagingapparatus according to claim 9, wherein the means for estimating thescattering feature has means for performing regular projection on atleast one predetermined vector in a spectrum space.
 11. The imagingapparatus according to claim 1, wherein the processor has means forgenerating a color image having a scattering feature by a living bodytissue as image information.
 12. The imaging apparatus according toclaim 11, wherein the means for generating the color image has means forgenerating a color image having an absorption feature by a living bodytissue as image information.
 13. The imaging apparatus according toclaim 11, wherein the means for generating a color image generates acolor image by synthesizing a scattering feature image and the otherimages.
 14. The imaging apparatus according to claim 11, wherein themeans for generating a color image generates an image and performsdisplay control such that a scattering feature image and the otherimages can be displayed simultaneously and/or in a switching manner. 15.The imaging apparatus according to claim 13, wherein images other thanthe scattering feature images are image information having absorptionfeatures by a living body tissue.
 16. The imaging apparatus according toclaim 13, wherein images other than the scattering feature imagescorrespond to images obtained under the illumination of white color. 17.The imaging apparatus according to claim 1, wherein the processor hasmeans for performing at least one spatial frequency filtering.
 18. Theimaging apparatus according to claim 17, wherein the means forperforming spatial frequency filtering is positioned in means forestimating, from at least one living body image, spectrums correspondingto positions and/or an area in the image.
 19. A living body scatteringimaging apparatus, comprising scattering feature computing means forcalculating a scattering feature from a living body tissue image andimage generating means for generating an image based on the scatteringfeature, wherein, in a case where the living body tissue is modeled intwo layers having a tissue surface layer and an internal layer otherthan the tissue surface layer, the scattering feature computing meanscalculates the scattering feature by applying a mapping of one or moreimage values or observation values based on the image values to ascattering feature space maximizing a change in scattering feature ofthe tissue surface layer under a condition minimizing an influence froma change in observation value occurring due to a change in opticalcharacteristic of the internal layer.
 20. The living body scatteringimaging apparatus according to claim 19, wherein the mapping is a linearmapping.
 21. The living body scattering imaging apparatus according toclaim 20, wherein the linear mapping is calculated by multiplediscrimination analysis in an observation value space and wherein aninter-class distribution and intra-class distribution in the multiplediscrimination analysis correspond to a difference in scattering featureof the tissue surface layer and a difference in optical characteristicof the internal layer.
 22. The living body scattering imaging apparatusaccording to claim 21, wherein the difference in optical characteristicof the internal layer includes a thickness, absorption characteristicand scattering characteristic of the internal layer.
 23. The living bodyscattering imaging apparatus according to claim 21, wherein thedifference in scattering characteristic of the tissue surface layer ismodeled by using changes in probability density distribution of anucleus diameter included in a living body tissue of the tissue surfacelayer.
 24. The living body scattering imaging apparatus according toclaim 23, wherein the model includes a Mie scattering model and a lightpropagation model.
 25. The living body scattering imaging apparatusaccording to claim 24, wherein the light propagation model is aMontecarlo model.
 26. The living body scattering imaging apparatusaccording to claim 21, wherein the intra-class distribution in themultiple discrimination analysis is estimated from the living bodytissue image.
 27. The living body scattering imaging apparatus accordingto claim 26, wherein the estimation of the intra-class distribution isperformed from an image of the blood vessel running within a living bodytissue.